A real-world evaluation of the implementation of NLP technology in abstract screening of a systematic review.

The laborious and time-consuming nature of systematic review production hinders the dissemination of up-to-date evidence synthesis. Well-performing natural language processing (NLP) tools for systematic reviews have been developed, showing promise to improve efficiency. However, the feasibility and value of these technologies have not been comprehensively demonstrated in a real-world review. We developed an NLP-assisted abstract screening tool that provides text inclusion recommendations, keyword highlights, and visual context cues. We evaluated this tool in a living systematic review on SARS-CoV-2 seroprevalence, conducting a quality improvement assessment of screening with and without the tool. We evaluated changes to abstract screening speed, screening accuracy, characteristics of included texts, and user satisfaction. The tool improved efficiency, reducing screening time per abstract by 45.9% and decreasing inter-reviewer conflict rates. The tool conserved precision of article inclusion (positive predictive value; 0.92 with tool vs. 0.88 without) and recall (sensitivity; 0.90 vs. 0.81). The summary statistics of included studies were similar with and without the tool. Users were satisfied with the tool (mean satisfaction score of 4.2/5). We evaluated an abstract screening process where one human reviewer was replaced with the tool's votes, finding that this maintained recall (0.92 one-person, one-tool vs. 0.90 two tool-assisted humans) and precision (0.91 vs. 0.92) while reducing screening time by 70%. Implementing an NLP tool in this living systematic review improved efficiency, maintained accuracy, and was well-received by researchers, demonstrating the real-world effectiveness of NLP in expediting evidence synthesis.

Protective effectiveness of previous SARS-CoV-2 infection and hybrid immunity against the omicron variant and severe disease: a systematic review and meta-regression.

BACKGROUND: The global surge in the omicron (B.1.1.529) variant has resulted in many individuals with hybrid immunity (immunity developed through a combination of SARS-CoV-2 infection and vaccination). We aimed to systematically review the magnitude and duration of the protective effectiveness of previous SARS-CoV-2 infection and hybrid immunity against infection and severe disease caused by the omicron variant. METHODS: For this systematic review and meta-regression, we searched for cohort, cross-sectional, and case-control studies in MEDLINE, Embase, Web of Science, ClinicalTrials.gov, the Cochrane Central Register of Controlled Trials, the WHO COVID-19 database, and Europe PubMed Central from Jan 1, 2020, to June 1, 2022, using keywords related to SARS-CoV-2, reinfection, protective effectiveness, previous infection, presence of antibodies, and hybrid immunity. The main outcomes were the protective effectiveness against reinfection and against hospital admission or severe disease of hybrid immunity, hybrid immunity relative to previous infection alone, hybrid immunity relative to previous vaccination alone, and hybrid immunity relative to hybrid immunity with fewer vaccine doses. Risk of bias was assessed with the Risk of Bias In Non-Randomized Studies of Interventions Tool. We used log-odds random-effects meta-regression to estimate the magnitude of protection at 1-month intervals. This study was registered with PROSPERO (CRD42022318605). FINDINGS: 11 studies reporting the protective effectiveness of previous SARS-CoV-2 infection and 15 studies reporting the protective effectiveness of hybrid immunity were included. For previous infection, there were 97 estimates (27 with a moderate risk of bias and 70 with a serious risk of bias). The effectiveness of previous infection against hospital admission or severe disease was 74·6% (95% CI 63·1-83·5) at 12 months. The effectiveness of previous infection against reinfection waned to 24·7% (95% CI 16·4-35·5) at 12 months. For hybrid immunity, there were 153 estimates (78 with a moderate risk of bias and 75 with a serious risk of bias). The effectiveness of hybrid immunity against hospital admission or severe disease was 97·4% (95% CI 91·4-99·2) at 12 months with primary series vaccination and 95·3% (81·9-98·9) at 6 months with the first booster vaccination after the most recent infection or vaccination. Against reinfection, the effectiveness of hybrid immunity following primary series vaccination waned to 41·8% (95% CI 31·5-52·8) at 12 months, while the effectiveness of hybrid immunity following first booster vaccination waned to 46·5% (36·0-57·3) at 6 months. INTERPRETATION: All estimates of protection waned within months against reinfection but remained high and sustained for hospital admission or severe disease. Individuals with hybrid immunity had the highest magnitude and durability of protection, and as a result might be able to extend the period before booster vaccinations are needed compared to individuals who have never been infected. FUNDING: WHO COVID-19 Solidarity Response Fund and the Coalition for Epidemic Preparedness Innovations.

Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance.

Background: Many serological assays to detect SARS-CoV-2 antibodies were developed during the COVID-19 pandemic. Differences in the detection mechanism of SARS-CoV-2 serological assays limited the comparability of seroprevalence estimates for populations being tested. Methods: We conducted a systematic review and meta-analysis of serological assays used in SARS-CoV-2 population seroprevalence surveys, searching for published articles, preprints, institutional sources, and grey literature between 1 January 2020, and 19 November 2021. We described features of all identified assays and mapped performance metrics by the manufacturers, third-party head-to-head, and independent group evaluations. We compared the reported assay performance by evaluation source with a mixed-effect beta regression model. A simulation was run to quantify how biased assay performance affects population seroprevalence estimates with test adjustment. Results: Among 1807 included serosurveys, 192 distinctive commercial assays and 380 self-developed assays were identified. According to manufacturers, 28.6% of all commercial assays met WHO criteria for emergency use (sensitivity [Sn.] >= 90.0%, specificity [Sp.] >= 97.0%). However, manufacturers overstated the absolute values of Sn. of commercial assays by 1.0% [0.1, 1.4%] and 3.3% [2.7, 3.4%], and Sp. by 0.9% [0.9, 0.9%] and 0.2% [−0.1, 0.4%] compared to third-party and independent evaluations, respectively. Reported performance data was not sufficient to support a similar analysis for self-developed assays. Simulations indicate that inaccurate Sn. and Sp. can bias seroprevalence estimates adjusted for assay performance; the error level changes with the background seroprevalence. Conclusions: The Sn. and Sp. of the serological assay are not fixed properties, but varying features depending on the testing population. To achieve precise population estimates and to ensure the comparability of seroprevalence, serosurveys should select assays with high performance validated not only by their manufacturers and adjust seroprevalence estimates based on assured performance data. More investigation should be directed to consolidating the performance of self-developed assays.

Global SARS-CoV-2 seroprevalence from January 2020 to April 2022: A systematic review and meta-analysis of standardized population-based studies.

BACKGROUND: Our understanding of the global scale of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection remains incomplete: Routine surveillance data underestimate infection and cannot infer on population immunity; there is a predominance of asymptomatic infections, and uneven access to diagnostics. We meta-analyzed SARS-CoV-2 seroprevalence studies, standardized to those described in the World Health Organization's Unity protocol (WHO Unity) for general population seroepidemiological studies, to estimate the extent of population infection and seropositivity to the virus 2 years into the pandemic. METHODS AND FINDINGS: We conducted a systematic review and meta-analysis, searching MEDLINE, Embase, Web of Science, preprints, and grey literature for SARS-CoV-2 seroprevalence published between January 1, 2020 and May 20, 2022. The review protocol is registered with PROSPERO (CRD42020183634). We included general population cross-sectional and cohort studies meeting an assay quality threshold (90% sensitivity, 97% specificity; exceptions for humanitarian settings). We excluded studies with an unclear or closed population sample frame. Eligible studies-those aligned with the WHO Unity protocol-were extracted and critically appraised in duplicate, with risk of bias evaluated using a modified Joanna Briggs Institute checklist. We meta-analyzed seroprevalence by country and month, pooling to estimate regional and global seroprevalence over time; compared seroprevalence from infection to confirmed cases to estimate underascertainment; meta-analyzed differences in seroprevalence between demographic subgroups such as age and sex; and identified national factors associated with seroprevalence using meta-regression. We identified 513 full texts reporting 965 distinct seroprevalence studies (41% low- and middle-income countries [LMICs]) sampling 5,346,069 participants between January 2020 and April 2022, including 459 low/moderate risk of bias studies with national/subnational scope in further analysis. By September 2021, global SARS-CoV-2 seroprevalence from infection or vaccination was 59.2%, 95% CI [56.1% to 62.2%]. Overall seroprevalence rose steeply in 2021 due to infection in some regions (e.g., 26.6% [24.6 to 28.8] to 86.7% [84.6% to 88.5%] in Africa in December 2021) and vaccination and infection in others (e.g., 9.6% [8.3% to 11.0%] in June 2020 to 95.9% [92.6% to 97.8%] in December 2021, in European high-income countries [HICs]). After the emergence of Omicron in March 2022, infection-induced seroprevalence rose to 47.9% [41.0% to 54.9%] in Europe HIC and 33.7% [31.6% to 36.0%] in Americas HIC. In 2021 Quarter Three (July to September), median seroprevalence to cumulative incidence ratios ranged from around 2:1 in the Americas and Europe HICs to over 100:1 in Africa (LMICs). Children 0 to 9 years and adults 60+ were at lower risk of seropositivity than adults 20 to 29 (p < 0.001 and p = 0.005, respectively). In a multivariable model using prevaccination data, stringent public health and social measures were associated with lower seroprevalence (p = 0.02). The main limitations of our methodology include that some estimates were driven by certain countries or populations being overrepresented. CONCLUSIONS: In this study, we observed that global seroprevalence has risen considerably over time and with regional variation; however, over one-third of the global population are seronegative to the SARS-CoV-2 virus. Our estimates of infections based on seroprevalence far exceed reported Coronavirus Disease 2019 (COVID-19) cases. Quality and standardized seroprevalence studies are essential to inform COVID-19 response, particularly in resource-limited regions.

Timeliness of reporting of SARS-CoV-2 seroprevalence results and their utility for infectious disease surveillance.

Seroprevalence studies have been used throughout the COVID-19 pandemic to monitor infection and immunity. These studies are often reported in peer-reviewed journals, but the academic writing and publishing process can delay reporting and thereby public health action. Seroprevalence estimates have been reported faster in preprints and media, but with concerns about data quality. We aimed to (i) describe the timeliness of SARS-CoV-2 serosurveillance reporting by publication venue and study characteristics and (ii) identify relationships between timeliness, data validity, and representativeness to guide recommendations for serosurveillance efforts. We included seroprevalence studies published between January 1, 2020 and December 31, 2021 from the ongoing SeroTracker living systematic review. For each study, we calculated timeliness as the time elapsed between the end of sampling and the first public report. We evaluated data validity based on serological test performance and correction for sampling error, and representativeness based on the use of a representative sample frame and adequate sample coverage. We examined how timeliness varied with study characteristics, representativeness, and data validity using univariate and multivariate Cox regression. We analyzed 1844 studies. Median time to publication was 154 days (IQR 64-255), varying by publication venue (journal articles: 212 days, preprints: 101 days, institutional reports: 18 days, and media: 12 days). Multivariate analysis confirmed the relationship between timeliness and publication venue and showed that general population studies were published faster than special population or health care worker studies; there was no relationship between timeliness and study geographic scope, geographic region, representativeness, or serological test performance. Seroprevalence studies in peer-reviewed articles and preprints are published slowly, highlighting the limitations of using the academic literature to report seroprevalence during a health crisis. More timely reporting of seroprevalence estimates can improve their usefulness for surveillance, enabling more effective responses during health emergencies.

Lessons from the pandemic: Responding to emerging zoonotic viral diseases-a Keystone Symposia report.

The COVID-19 pandemic caught the world largely unprepared, including scientific and policy communities. On April 10-13, 2022, researchers across academia, industry, government, and nonprofit organizations met at the Keystone symposium "Lessons from the Pandemic: Responding to Emerging Zoonotic Viral Diseases" to discuss the successes and challenges of the COVID-19 pandemic and what lessons can be applied moving forward. Speakers focused on experiences not only from the COVID-19 pandemic but also from outbreaks of other pathogens, including the Ebola virus, Lassa virus, and Nipah virus. A general consensus was that investments made during the COVID-19 pandemic in infrastructure, collaborations, laboratory and manufacturing capacity, diagnostics, clinical trial networks, and regulatory enhancements-notably, in low-to-middle income countries-must be maintained and strengthened to enable quick, concerted responses to future threats, especially to zoonotic pathogens.

Deploying wearable sensors for pandemic mitigation: A counterfactual modelling study of Canada's second COVID-19 wave.

Wearable sensors can continuously and passively detect potential respiratory infections before or absent symptoms. However, the population-level impact of deploying these devices during pandemics is unclear. We built a compartmental model of Canada's second COVID-19 wave and simulated wearable sensor deployment scenarios, systematically varying detection algorithm accuracy, uptake, and adherence. With current detection algorithms and 4% uptake, we observed a 16% reduction in the second wave burden of infection; however, 22% of this reduction was attributed to incorrectly quarantining uninfected device users. Improving detection specificity and offering confirmatory rapid tests each minimized unnecessary quarantines and lab-based tests. With a sufficiently low false positive rate, increasing uptake and adherence became effective strategies for scaling averted infections. We concluded that wearable sensors capable of detecting presymptomatic or asymptomatic infections have potential to help reduce the burden of infection during a pandemic; in the case of COVID-19, technology improvements or supporting measures are required to keep social and resource costs sustainable.

SARS-CoV-2 infection in Africa: a systematic review and meta-analysis of standardised seroprevalence studies, from January 2020 to December 2021.

INTRODUCTION: Estimating COVID-19 cumulative incidence in Africa remains problematic due to challenges in contact tracing, routine surveillance systems and laboratory testing capacities and strategies. We undertook a meta-analysis of population-based seroprevalence studies to estimate SARS-CoV-2 seroprevalence in Africa to inform evidence-based decision making on public health and social measures (PHSM) and vaccine strategy. METHODS: We searched for seroprevalence studies conducted in Africa published 1 January 2020-30 December 2021 in Medline, Embase, Web of Science and Europe PMC (preprints), grey literature, media releases and early results from WHO Unity studies. All studies were screened, extracted, assessed for risk of bias and evaluated for alignment with the WHO Unity seroprevalence protocol. We conducted descriptive analyses of seroprevalence and meta-analysed seroprevalence differences by demographic groups, place and time. We estimated the extent of undetected infections by comparing seroprevalence and cumulative incidence of confirmed cases reported to WHO. PROSPERO: CRD42020183634. RESULTS: We identified 56 full texts or early results, reporting 153 distinct seroprevalence studies in Africa. Of these, 97 (63%) were low/moderate risk of bias studies. SARS-CoV-2 seroprevalence rose from 3.0% (95% CI 1.0% to 9.2%) in April-June 2020 to 65.1% (95% CI 56.3% to 73.0%) in July-September 2021. The ratios of seroprevalence from infection to cumulative incidence of confirmed cases was large (overall: 100:1, ranging from 18:1 to 954:1) and steady over time. Seroprevalence was highly heterogeneous both within countries-urban versus rural (lower seroprevalence for rural geographic areas), children versus adults (children aged 0-9 years had the lowest seroprevalence)-and between countries and African subregions. CONCLUSION: We report high seroprevalence in Africa suggesting greater population exposure to SARS-CoV-2 and potential protection against COVID-19 severe disease than indicated by surveillance data. As seroprevalence was heterogeneous, targeted PHSM and vaccination strategies need to be tailored to local epidemiological situations.

Adapting Serosurveys for the SARS-CoV-2 Vaccine Era.

Population-level immune surveillance, which includes monitoring exposure and assessing vaccine-induced immunity, is a crucial component of public health decision-making during a pandemic. Serosurveys estimating the prevalence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibodies in the population played a key role in characterizing SARS-CoV-2 epidemiology during the early phases of the pandemic. Existing serosurveys provide infrastructure to continue immune surveillance but must be adapted to remain relevant in the SARS-CoV-2 vaccine era. Here, we delineate how SARS-CoV-2 serosurveys should be designed to distinguish infection- and vaccine-induced humoral immune responses to efficiently monitor the evolution of the pandemic. We discuss how serosurvey results can inform vaccine distribution to improve allocation efficiency in countries with scarce vaccine supplies and help assess the need for booster doses in countries with substantial vaccine coverage.

Adapting serosurveys for the SARS-CoV-2 vaccine era

Population-level immune surveillance, which includes monitoring exposure and assessing vaccine-induced immunity, is a crucial component of public health decision-making during a pandemic. Serosurveys estimating the prevalence of SARS-CoV-2 antibodies in the population played a key role in characterizing SARS-CoV-2 epidemiology during the early phases of the pandemic. Existing serosurveys provide infrastructure to continue immune surveillance, but must be adapted to remain relevant in the SARS-CoV-2 vaccine era. Here, we delineate how SARS-CoV-2 serosurveys should be designed to distinguish infection- and vaccine-induced humoral immune responses to efficiently monitor the evolution of the pandemic. We discuss how serosurvey results can inform vaccine distribution to improve allocation efficiency in countries with scarce vaccine supplies and help assess the need for booster doses in countries with substantial vaccine coverage.